ABC News

In an interview with ABC's Diane Sawyer and George Stephanopoulos during ABC's special inauguration day coverage this morning, former Secretary of State Colin Powell lashed out at people in the Republican Party who spent the last four years spreading "birther nonsense" and other "things that demonize the president," calling on GOP leaders to denounce such talk - publicly.

"Republicans have to stop buying into things that demonize the president. I mean, why aren't Republican leaders shouting out about all this birther nonsense and all these other things? They should speak out. This is the kind of intolerance that I've been talking about where these idiot presentations continue to be made and you don't see the senior leadership of the party say, 'No, that's wrong.' In fact, sometimes by not speaking out, they're encouraging it. And the base keeps buying the stuff.

"And it's killing the base of the party. I mean, 26 percent favorability rating for the party right now. It ought to be telling them something. So, instead of attacking me or whoever speaks like I do, look in the mirror and realize, 'How are we going to win the next election?"

But Powell, who served as National Security Adviser under President Ronald Reagan, Chairman of the Joint Chiefs of Staff under George H.W. Bush and as the nation's top diplomat under President George W. Bush, didn't stop there.

Tune in to the ABC News.com Live page on Monday morning starting at 9:30 a.m. EST for all-day live streaming video coverage of Inauguration 2013: Barack Obama. Live coverage will also be available on the ABC News iPad App and mobile devices.

"The Republican Party ought to be out there not restricting voting by voter ID, but saying we want everybody to vote," he told Sawyer and Stephanopoulos. "It's a party that has to stop saying, 'We are going to appeal to you with new messages.' You need policies - the country is becoming more minority."

Powell, who endorsed President Obama's reelection bid in October, said that his critiques of the GOP have left some wondering, "Why are you still a Republican?"

Because, Powell said, "I grew up under Ronald Reagan and Cap Weinberger and George Schultz and George Herbert Walker Bush - that's the Republican party I know - the Howard Bakers of the world, and I think we've drifted from that. And I don't think there's anything wrong with trying to drift a little bit back. Not because it's just good to be moderate, but because that's where the American people are. They lost an election - two."

But Powell also said President Obama needed to do a better job, during his second, term of reaching out to members of Congress. And as for the president's second inaugural address, Powell said, "I hope he can, through his own example, restore a sense of civility in the country."

He said that today was a day for members of both parties, "not to scream and argue with each other, but to come together."